chore: Fix configuration of EhCache 3 #3118

pj892031 · 2023-10-02T14:15:39Z

Description

This PR fixes the upgrade of EhCache (see #3098). There is one cache (invalidatedJwtTokens) that persists data to be stale over restarts. The issue here is about the new implementation. EhCache creates a random access file and its channel. Then it creates off-heap caches (by linking data into memory), see:

org.ehcache.shadow.org.terracotta.offheapstore.disk.paging.MappedPageSource:

public synchronized MappedPage allocate(int size, boolean thief, boolean victim, OffHeapStorageArea owner) {
    //...
    MappedByteBuffer buffer = (MappedByteBuffer)this.channel.execute((fc) -> {
        return fc.map(MapMode.READ_WRITE, address, (long)size);
    });
    MappedPage page = new MappedPage(buffer);
    this.pages.put(page, address);
    //...
}

In other OS than z/OS is possible to create multiple buffers at the same time (<0;1024), <1024;2048), etc.), but z/OS limits somehow this feature. It seems there are possible to allocate multiple buffers as well but with a limitation (blocks cannot be near each other, but with a non-specified gap).

The solution is to create just one buffer and avoid parallel access to this file. It is done by setting disk segments to 1.

Log:

2023-09-29 14:28:45.492 <ZWEAGW1:main:67174564> ZWESVUSR WARN  (o.e.s.o.t.o.d.p.MappedPageSource) Mapping a new file section failed
java.io.IOException: EDC5121I Invalid argument. (errno2=0x0736033B)
	at sun.nio.ch.FileChannelImpl.map0(Native Method)
	at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:946)
	at org.ehcache.shadow.org.terracotta.offheapstore.disk.paging.MappedPageSource.lambda$allocate$3(MappedPageSource.java:165)
	at org.ehcache.shadow.org.terracotta.offheapstore.disk.paging.MappedPageSource$$Lambda$1006/00000000201F94D0.apply(Unknown Source)
	at org.ehcache.shadow.org.terracotta.offheapstore.util.ReopeningInterruptibleChannel.execute(ReopeningInterruptibleChannel.java:53)
	at org.ehcache.shadow.org.terracotta.offheapstore.disk.paging.MappedPageSource.allocate(MappedPageSource.java:165)
	at org.ehcache.shadow.org.terracotta.offheapstore.disk.paging.MappedPageSource.allocate(MappedPageSource.java:46)
	at org.ehcache.shadow.org.terracotta.offheapstore.OffHeapHashMap.allocateTable(OffHeapHashMap.java:1264)
	at org.ehcache.shadow.org.terracotta.offheapstore.OffHeapHashMap.<init>(OffHeapHashMap.java:201)
	at org.ehcache.shadow.org.terracotta.offheapstore.AbstractLockedOffHeapHashMap.<init>(AbstractLockedOffHeapHashMap.java:65)
	at org.ehcache.shadow.org.terracotta.offheapstore.AbstractOffHeapClockCache.<init>(AbstractOffHeapClockCache.java:76)
	at org.ehcache.shadow.org.terracotta.offheapstore.disk.persistent.AbstractPersistentOffHeapCache.<init>(AbstractPersistentOffHeapCache.java:43)
	at org.ehcache.shadow.org.terracotta.offheapstore.disk.persistent.PersistentReadWriteLockedOffHeapClockCache.<init>(PersistentReadWriteLockedOffHeapClockCache.java:36)
	at org.ehcache.impl.internal.store.disk.factories.EhcachePersistentSegmentFactory$EhcachePersistentSegment.<init>(EhcachePersistentSegmentFactory.java:78)
	at org.ehcache.impl.internal.store.disk.factories.EhcachePersistentSegmentFactory.newInstance(EhcachePersistentSegmentFactory.java:65)
	at org.ehcache.impl.internal.store.disk.factories.EhcachePersistentSegmentFactory.newInstance(EhcachePersistentSegmentFactory.java:42)
	at org.ehcache.shadow.org.terracotta.offheapstore.concurrent.AbstractConcurrentOffHeapMap.<init>(AbstractConcurrentOffHeapMap.java:106)
	at org.ehcache.shadow.org.terracotta.offheapstore.concurrent.AbstractConcurrentOffHeapCache.<init>(AbstractConcurrentOffHeapCache.java:43)
	at org.ehcache.shadow.org.terracotta.offheapstore.disk.persistent.AbstractPersistentConcurrentOffHeapCache.<init>(AbstractPersistentConcurrentOffHeapCache.java:52)
	at org.ehcache.impl.internal.store.disk.EhcachePersistentConcurrentOffHeapClockCache.<init>(EhcachePersistentConcurrentOffHeapClockCache.java:52)
	at org.ehcache.impl.internal.store.disk.OffHeapDiskStore.createBackingMap(OffHeapDiskStore.java:268)
	at org.ehcache.impl.internal.store.disk.OffHeapDiskStore.getBackingMap(OffHeapDiskStore.java:165)
	at org.ehcache.impl.internal.store.disk.OffHeapDiskStore.access$600(OffHeapDiskStore.java:90)
	at org.ehcache.impl.internal.store.disk.OffHeapDiskStore$Provider.init(OffHeapDiskStore.java:441)
	at org.ehcache.impl.internal.store.disk.OffHeapDiskStore$Provider.initStore(OffHeapDiskStore.java:437)
	at org.ehcache.impl.internal.store.disk.OffHeapDiskStore$Provider.initAuthoritativeTier(OffHeapDiskStore.java:483)
	at org.ehcache.impl.internal.store.tiering.TieredStore$Provider.initStore(TieredStore.java:549)
	at org.ehcache.core.EhcacheManager$7.init(EhcacheManager.java:514)
	at org.ehcache.core.StatusTransitioner.runInitHooks(StatusTransitioner.java:130)
	at org.ehcache.core.StatusTransitioner.access$000(StatusTransitioner.java:33)
	at org.ehcache.core.StatusTransitioner$Transition.succeeded(StatusTransitioner.java:189)
	at org.ehcache.core.EhcacheBase.init(EhcacheBase.java:567)
	at org.ehcache.core.EhcacheManager.createCache(EhcacheManager.java:275)
	at org.ehcache.core.EhcacheManager.init(EhcacheManager.java:589)
	at org.ehcache.jsr107.EhcacheCachingProvider.createCacheManager(EhcacheCachingProvider.java:159)
	at org.ehcache.jsr107.EhcacheCachingProvider.getCacheManager(EhcacheCachingProvider.java:134)
	at org.ehcache.jsr107.EhcacheCachingProvider.getCacheManager(EhcacheCachingProvider.java:85)
	at org.springframework.cache.jcache.JCacheManagerFactoryBean.afterPropertiesSet(JCacheManagerFactoryBean.java:83)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1863)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1800)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$221/00000000320E9E30.getObject(Unknown Source)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:307)
	at org.zowe.apiml.gateway.config.CacheConfig$$EnhancerBySpringCGLIB$$5516e61a.cacheManagerFactoryBean(<generated>)
	at org.zowe.apiml.gateway.config.CacheConfig.cacheManager(CacheConfig.java:61)
	at org.zowe.apiml.gateway.config.CacheConfig$$EnhancerBySpringCGLIB$$5516e61a.CGLIB$cacheManager$2(<generated>)
	at org.zowe.apiml.gateway.config.CacheConfig$$EnhancerBySpringCGLIB$$5516e61a$$FastClassBySpringCGLIB$$8ca44967.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
	at org.zowe.apiml.gateway.config.CacheConfig$$EnhancerBySpringCGLIB$$5516e61a.cacheManager(<generated>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:508)
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:648)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:481)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$221/00000000320E9E30.getObject(Unknown Source)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:911)
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788)
	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:224)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$221/00000000320E9E30.getObject(Unknown Source)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1609)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1573)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1462)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1349)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:911)
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788)
	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:224)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$221/00000000320E9E30.getObject(Unknown Source)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1609)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1573)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1462)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1349)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:911)
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788)
	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:224)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$221/00000000320E9E30.getObject(Unknown Source)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:921)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:147)
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:731)
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:307)
	at org.zowe.apiml.gateway.GatewayApplication.main(GatewayApplication.java:57)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:508)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
	at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
2023-09-29 14:28:45.755 <ZWEAGW1:main:67174564> ZWESVUSR ERROR (o.e.c.StatusTransitioner) Initialize failed.
2023-09-29 14:28:45.759 <ZWEAGW1:main:67174564> ZWESVUSR WARN  (o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext) Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cacheServiceController' defined in URL [jar:file:/ZOWE/runtime/components/gateway/bin/gateway-service-lite.jar!/BOOT-INF/classes!/org/zowe/apiml/gateway/controllers/CacheServiceController.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'serviceCacheEvictor' defined in URL [jar:file:/ZOWE/runtime/components/gateway/bin/gateway-service-lite.jar!/BOOT-INF/classes!/org/zowe/apiml/gateway/cache/ServiceCacheEvictor.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'serviceAuthenticationServiceImpl' defined in URL [jar:file:/ZOWE/runtime/components/gateway/bin/gateway-service-lite.jar!/BOOT-INF/classes!/org/zowe/apiml/gateway/security/service/ServiceAuthenticationServiceImpl.class]: Unsatisfied dependency expressed through constructor parameter 5; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cacheManager' defined in class path resource [org/zowe/apiml/gateway/config/CacheConfig.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cache.CacheManager]: Factory method 'cacheManager' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cacheManagerFactoryBean' defined in class path resource [org/zowe/apiml/gateway/config/CacheConfig.class]: Invocation of init method failed; nested exception is org.ehcache.StateTransitionException: Cache 'invalidatedJwtTokens' creation in EhcacheManager failed.

Linked to # (issue)
Part of the # (epic)

Type of change

Please delete options that are not relevant.

fix: Bug fix (non-breaking change which fixes an issue)
feat: New feature (non-breaking change which adds functionality)
docs: Change in a documentation
refactor: Refactor the code
chore: Chore, repository cleanup, updates the dependencies.
BREAKING CHANGE or !: Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

My code follows the style guidelines of this project
PR title conforms to commit message guideline ## Commit Message Structure Guideline
I have commented my code, particularly in hard-to-understand areas. In JS I did provide JSDoc
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
The java tests in the area I was working on leverage @nested annotations
Any dependent changes have been merged and published in downstream modules

For more details about how should the code look like read the Contributing guideline

Signed-off-by: Pavel Jareš <[email protected]>

sonarcloud · 2023-10-02T14:52:08Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

The version of Java (11.0.20) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

fix

904f022

Signed-off-by: Pavel Jareš <[email protected]>

pull-request-size bot added the size/XS label Oct 2, 2023

pj892031 requested review from pablocarle and achmelo October 2, 2023 14:15

pablocarle approved these changes Oct 2, 2023

View reviewed changes

pj892031 marked this pull request as ready for review October 2, 2023 14:58

pj892031 merged commit 4924a66 into v1.x.x Oct 2, 2023
27 checks passed

delete-merged-branch bot deleted the reboot/fix_ehcache3_v1 branch October 2, 2023 15:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: Fix configuration of EhCache 3 #3118

chore: Fix configuration of EhCache 3 #3118

pj892031 commented Oct 2, 2023

sonarcloud bot commented Oct 2, 2023

chore: Fix configuration of EhCache 3 #3118

chore: Fix configuration of EhCache 3 #3118

Conversation

pj892031 commented Oct 2, 2023

Description

Type of change

Checklist:

sonarcloud bot commented Oct 2, 2023